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Executive Summary 


With a national reputation for high standards and a rigorous assessment system, Massachusetts now faces 
a critical transition. The Massachusetts Board of Elementary and Secondary Education’s December 2015 
decision to develop a new state assessment (Next-Generation MCAS) brought with it the opportunity to 
improve the 2011 state standards for ELA/Literacy and mathematics, five years after their adoption. 
Massachusetts has revised its standards several times since 1995, when the first curriculum frameworks 
were adopted. For revisions to the standards to inform the design of the state assessment, however, the 
timeline for the revision process had to be condensed to occur from January 2016-March/April 2017. 


To ensure that both the revised standards and the new assessment system are high quality, the Department 
has continued its longstanding practice of soliciting and using stakeholders’ content expertise and 
knowledge of standards implementation to inform the revision of the standards and assessment system. 
Balancing the various opportunities and constraints presented by this timeline, the Department initiated a 
three-phase process for standards revisions, as described below: 


e Phase 1 (January-July 2016): The Department engaged stakeholders through various methods, 
gathered recommendations for revisions, began applying the suggestions to the standards document as 
proposed changes and continued an iterative process of receiving feedback from stakeholders; 


e Phase 2 (August-October 2016): The Department refined revisions and engaged content advisors to 
inform proposed revisions for Board consideration and discussion in October; and 


e Phase 3 (November 2016-March/April 2017): The Department will incorporate Board feedback on 
proposed revisions from BESE’s October meeting and present drafts reflecting that feedback to the 
Board in November for its vote to release the drafts of the Frameworks documents for public 
comment. From December through March or April, the Department will conduct a public comment 
period, synthesize public comment, make final revisions and bring the final ELA/Literacy and 
Mathematics Frameworks documents before the Board for a vote of formal adoption. 


Abt Associates has prepared this report to inform the Board about the standards review process conducted 
to date, and to assist the Board in its efforts to consider and incorporate feedback and guidance obtained 
from key stakeholders. In the following sections, we summarize the standards revision process, describe 
the stakeholder feedback and from whom it was collected, provide an analysis of the feedback and 
describe how the major themes for the revisions in ELA/literacy and mathematics emerged. 


Phase 1: Methods for Collecting Stakeholder Feedback 


Phase 1 began with the Department conducting outreach to establish both a statewide English Language 
Arts (ELA) and Literacy and Mathematics Standards Review Panel (Review Panel) and an online survey. 
The English Language Arts/Literacy and Mathematics Standards Review Panel’s members were charged 
with developing evidence-based recommendations to refine the 2011 ELA/Literacy and Mathematics 
standards based on lessons learned during Massachusetts’ implementation over the previous five years. In 
early February 2016, ESE selected 43 Panelists from across the state; 20 Panelists brought expertise in 
ELA/Literacy and 23 in mathematics. In addition, the Department created and posted an online survey to 
solicit public input on the ELA/Literacy and Mathematics Curriculum Frameworks. Aligned with the 
purpose of the Review Panel, the survey was designed to provide educators across the Commonwealth an 
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opportunity to share lessons learned from implementing the 2011 ELA/literacy and mathematics 
standards. 


The survey and Review Panel each yielded valuable feedback about potential revisions to the 
ELA/literacy and mathematics standards. Over the course of four different day-long meetings between 
February and July, members of the Review Panel met to review feedback from the online survey and 
discuss potential revisions to the standards. Department staff asked Panel members to provide feedback 
on several topics for potential revisions in ELA/literacy, including: the quantity and rigor of pre- 
kindergarten through grade 5 standards, the clarity and appropriateness of language standards, especially 
in the early and middle years, the level of attention paid to motor processes of writing, the modification of 
the Massachusetts-specific standards focused on identifying and analyzing literary works and on writing 
in different literary genres to make the genres less limited and/or the grade level expectations less 
specific, clarification of the coherent progression of middle school reading standards as well as clarity 
about the integration of various media, the modification of the “Text Types and Purposes” cluster of 
writing standards to prevent the standards from being too rigid in defining modes of writing, modification 
of the standards relating to narrative writing, and the standards for literacy in the content areas to make 
clear the role of ELA teachers working with informational text related to other disciplines, and 
consideration of removing or reducing the references to United States history in the ELA standards. 


In mathematics, the Department asked Panel members to provide feedback on several topics for potential 
revision, including: 1) whether or not there were aspects of the PK-5 standards that could be reduced, 2) 
if K-3 standards were missing some critical components (such as time and money, patterns), 3) whether 
there were aspects of the 6-8 standards that could be reduced, 4) if adjustments should be made to critical 
content in the grades 6-8 standards, and 5) how the scope of high school standards might be reduced (e.g. 
the scope of Algebra I, concepts related to complex numbers and matrices, whether the scope of statistics 
be reduced and/or moved to different courses). 


Phase 2: Refine Revisions and Engage Content Advisors 


In Phase 2 of the standards review process, the Department documented and summarized survey 
responses and Review Panel feedback from Phase 1, followed by the development of a systematic 
approach to vetting and refining the proposed revisions in preparation for Board consideration. To lead 
this phase of the process, in July 2016, the Department engaged two experienced and widely-respected 
former staff members, Barbara Libby and Susan Wheltle to lead the revision process in mathematics and 
ELA/literacy, respectively. Barbara Libby is the retired Director of the ESE Office of Science, 
Technology/Engineering and Mathematics and Susan Wheltle is the retired Director of the Office of 
Literacy and Humanities. Libby and Wheltle led ESE’s previous frameworks writing and review panels in 
1995-97, 2000-2004, and 2007-2010. 


Libby and Wheltle, together with current members of the ELA/literacy and mathematics staff from the 
Department, reviewed the Phase 1 revisions proposed for mathematics and ELA/literacy, respectively. 
The ESE ELA/literacy and mathematics teams then identified both major issues and detailed changes 
being considered for recommendation in each content area. 


In ELA/literacy, the Department team identified the following major categories in the revisions: 
e Integrated the content from two PK-12 Reading and Writing standards added by Massachusetts in 
2010 into other standards, thereby reducing the overall number of standards 
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e Edited standards to create greater clarity and smoother progressions in complexity, PK-12 
e Made explicit cross-references among standards to increase coherence 


e Revised and expanded the glossary to provide a stronger resource for definitions of terms related to 
early literacy, writing, reading, language, and Standard English conventions 


In math, Department staff identified the following major categories for possible revisions emerging from 
Phase 1 activities: 


e Improvements in clarity and inclusion of definitions for key mathematical terms to ensure consistency 
in the language of the standards; 


e The development of customized descriptions of the eight PK-12 Standards for Mathematical Practice 
by grade-spans, PK-5; 6-8; and high school level; 


e Additional guidance that presents middle and high school options for course sequencing, including 
pathways to Algebra I in grade 8 and to calculus by grade 12. 


As staff at the Department began to refine the revisions to standards that had been proposed by Panelists 
and survey respondents in Phase 1, they recognized the potential value of having this work reviewed by 
an additional set of advisors who possessed deep content area expertise as well as prior experience in 
reviewing and supporting implementation of state standards. In August 2016, the Department established 
two groups of content advisors, one for ELA/literacy and one for mathematics, whose members consisted 
of educators from Massachusetts public and private institutions of higher education, K-12 educators and 
individuals with experience supporting elementary and secondary educators’ instruction. 


The content advisors met for two separate day-long meetings (one for ELA/literacy and one for 
mathematics) on September 15 and 16, 2016. The content advisors were asked to review draft revisions 
and draw on their own content expertise and professional experience to provide guidance and suggestions. 
ELA/literacy advisors made the following recommendations: maintain the strength of the standards, 
improve the clarity of some standards, support implementation with links to guidance documents and 
resources, improve information about text complexity, preserve strong progressions in Reading Literature 
Standards, highlight integration in types of writing, make stronger connections among the Language, 
Writing and Reading standards, and encourage teaching of literacy in subjects other than English. 


Similar to the ELA/literacy advisors, the mathematics advisors made the following recommendations: 
maintain the strength of the standards, improve the clarity of some standards, and support implementation 
by linking existing ESE resources and guidance documents to digital versions of the revised Frameworks. 
In addition, mathematics advisors, had content-specific suggestions, including: keep the Math Pathways 
guidance document in the revised Framework, keep the (+) standards in the Model High School courses, 
no need to develop a model statistics or calculus course now, add guidance and rationale for statistics, 
keep current language for “fluently” and “know by memory,” clarify the Ratio, Rate and Proportional 
Reasoning Progression in grade 6-7, redesign the grade-span descriptions of the 8 Standards for 
Mathematical Practice. 


The Department’s ELA/literacy and mathematics standards review teams have incorporated the content 
advisors’ feedback into the summaries of revisions to be presented to the Board at its October 2016 
meeting. The summaries will be presented in two documents: one that presents major issues and one that 
presents the details of revisions by grade level with the original and revised wording of the standards 
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followed by rationales for the changes. The summary documents, therefore, will reflect the work of ESE’s 
ELA/literacy and mathematics standards review teams, led by Wheltle and Libby, to synthesize and 
incorporate the deliberations of the Review Panels and content advisors conducted during Phases 1 and 2 
of the review process. 


Phase 3: Soliciting Public Comment and Adopting Revisions 


The 2016 standards revision process, while not a wholesale revision of standards, has provided a strategic 
opportunity to make improvements to the standards based on five years’ of experience of implementation. 
In Phase 3, The Department will incorporate Board feedback on proposed revisions from BESE’s October 
2016 meeting and will present full drafts to the Board in November for its vote to release the drafts of the 
Frameworks documents for public comment. From December 2016 through March or April 2017, the 
Department will conduct a public comment period, synthesize public comment, make final revisions and 
bring the completed ELA/Literacy and Mathematics Frameworks documents to the Board for adoption. 
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1. Introduction and Overview 


In December 2015, following a vote to develop a new state assessment system (Next-Generation MCAS), 
the Massachusetts Board of Elementary and Secondary Education (BESE) directed the Department of 
Elementary and Secondary Education (ESE) to consult with K-16 educators, curriculum specialists, and 
others to help identify recommendations for potential improvements to the Massachusetts English 
Language Arts (ELA) and Literacy and Mathematics Curriculum Frameworks based on lessons learned 
from implementation of the standards over the previous five years. As part of the development of the 
Next-Generation MCAS, the Board established the Assessment Oversight Committee for school year 
2015-16, which was charged with providing direction for the project, monitoring its progress, receiving 
recommendations from the various workgroups, and making periodic reports to the full BESE. The 
oversight committee was chaired by Board member Roland Fryer, and included Penny Noyce (vice chair), 
Secretary of Education James Peyser, Board Chair Paul Sagan, and Donald Willyard; it provided 
coordination of both the standards review and assessment development efforts. 


The development of the new assessment presented the Board with a logical opportunity to solicit input on 
potential improvements to the standards based on the first five years of implementation of the current 
standards. Since the Department wished to use revised and improved standards as the basis for 
development of Next-Generation MCAS items, the standards review process needed to be completed 
according to an expedited timeline. 


Balancing the various opportunities and constraints presented by this timeline, the Department initiated a 
process for standards revisions, as described below: 


e Phase 1 (January-July 2016): The Department engaged stakeholders through various methods, 
gathered recommendations for revisions, began applying the suggestions to the standards document as 
proposed changes and continued an iterative process of receiving of feedback from stakeholders; 


e Phase 2 (August-October 2016): The Department refined revisions and engaged content advisors to 
inform proposed revisions for Board consideration and discussion in October; and 


e Phase 3 (November 2016-March/April 2017): The Department will incorporate Board feedback on 
proposed revisions from BESE’s October 2016 meeting and will present drafts reflecting that 
feedback to the Board for its November 2016 vote to release the drafts of the Frameworks documents 
for public comment. From December 2016 through March or April 2017, the Department will 
conduct a public comment period, synthesize public comment, make final revisions and bring the 
final ELA/Literacy and Mathematics Frameworks documents before the Board for a vote of formal 
adoption. 
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What are Standards and Curriculum Frameworks? 


The Massachusetts Education Reform Act of 1993 defined standards as a set of statewide 
educational goals for students in grades kindergarten through twelve that clearly set forth the 
skills, competencies and knowledge expected to be possessed by all students at the 
conclusion of individual grades or clusters of grades. The Department developed the 
Curriculum Frameworks documents to provide guidance for the implementation of standards. 
The Frameworks begin with a vision statement, provide Guiding Principles for effective 
programs, describe the structure, organization and key features of the standards and present 
the learning standards at each grade level or span. The Frameworks documents also include 
materials to help educators implement the standards. The Department created the diagram 
below to describe the role standards play in a standards based system. 


A Standards-Based System 


Licensure 


Leadership ls 


Assessment Curriculum 


/ STUDENTS ; 
Intervention, 


Remediation Instruction 


Massachusetts Department of Elementary and Secondary Education 


The diagram indicates that standards establish learning goals, but do not dictate how 
educators may teach to achieve these goals with their students. The standards and state 
assessment, indicated above the dotted line in the diagram above, are determined at the 
state level. The topics that appear below the dotted line: how teachers teach a given topic 
(curriculum and instruction) and assess students’ progress toward meeting the learning 
standards on an ongoing basis (assessment), are determined at the local level. 
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Brief History of Massachusetts Curriculum Frameworks 


Massachusetts was one of the first states to implement standards-based reform with the 
enactment of the Massachusetts Education Reform Act (MERA) in 1993 and is widely 
considered a national leader in developing rigorous curriculum standards with a clear 
progression of knowledge and skills over time. In collaboration with educators across the 
state, the Department first developed curriculum standards and frameworks in mathematics 
and ELA that were adopted in 1995 and 1997 respectively. ESE subsequently engaged 
review panels in revising the Frameworks that were adopted in 2000 (mathematics) and 2001 
(ELA). Supplements were adopted in 2004, and a new round of review and revision took 
place between 2007 and 2010. 


In 2009, the National Governors Association (NGA) and the Council of Chief State School 
Officers (CCSSO) started their effort to write a set of multi-state standards, which became 
known as the Common Core State Standards. The work of Massachusetts’ review panels was 
deliberately coordinated with the NGA's and CCSSO's effort. Massachusetts then played a 
key role in the development of these standards. Six of ESE’s staff members were invited by 
CCSSO and NGA to serve on the Common Core State Standards Development Teams; and 
14 Massachusetts scholars and educators contributed their expertise to reviewing successive 
drafts in order to shape Massachusetts’ written comments to CCSSO and NGA. 


In June 2010, the Board voted to adopt the Common Core State Standards (CCSS) for 
English Language Arts and Literacy and Mathematics. As part of that adoption, 
Massachusetts was required to accept 100 percent of the CCSS verbatim, but was allowed to 
add 15 percent in additional standards. Massachusetts opted to add 22 of its own standards 
in mathematics and 24 Massachusetts-specific standards in ELA. 


In June 2016, in response to discussions in the Assessment Oversight Committee, the Executive Office of 
Education (EOE) retained Abt Associates to provide independent documentation of the standards review 
process and to provide limited research support to the Department in response to staffing changes.’ 


With a national reputation for high standards and a rigorous assessment system, Massachusetts now faces 
a critical transition. To ensure that both the updated standards and the new assessment system are high 
quality, the Department has continued its longstanding practice of soliciting and using stakeholders’ 
content expertise and knowledge of standards implementation to inform the revision of the standards and 
assessment system. Abt Associates has prepared this report to inform the Board of the standards review 
process that has been conducted to date and to assist the Board in considering and incorporating this 
feedback and guidance from key stakeholders. In the following sections, we summarize the standards 
revision process, describe the stakeholder feedback and from whom it was collected, provide an analysis 
of the feedback and describe how the major themes for the revisions in ELA and mathematics emerged. 


' In the summer of 2016, the following staff left ESE: Senior Associate Commissioner for Curriculum and 


Instruction, Acting Director of Science, Technology/Engineering and Mathematics (STEM) and Math Content 
Lead for the Model Curriculum Project, all of whom had been leaders of the standards revision process. 
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Massachusetts Start and End 


Standards Review Process 


i mm an Fetes tra 20001 || | || 


Department solicits applications for Review Panel Early Jan. 2016 

Review Panel responses due Late Jan. 2016 i 

Review Panel member selected Early Feb. 2016 Ea 

Online survey posted Early Feb. 2016 Na 

First Review Panel meeting Feb. 26, 2016 A 

Second Review Panel meeting Apr. 13, 2016 Fal 

Third Review Panel meeting Jun. 3, 2016 ia 

Fourth Review Panel meeting Jul. 7, 2016 Fil 
es 

Online survey closed Sept. 2016 4 

ELA/Literacy and Math content advisors meetings Sept. 15-16, 2016 J 

Presentation to BESE of standards revision Oct. 24, 2016 


process and major categories for revisions 
informed by Phases 1 and 2 


a S27 | | 00) | 


Department presents drafts of proposed revisions Nov.-Mar. 2017 
to the Board for a vote to release revisions for 

public comment; conduct public comment period, 

synthesize public comment; make final revisions 

and bring before the Board to vote. 
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2. The Standards Revision Process 


Phase 1 of the revision process began with the Department engaging in two main activities: establishing a 
Review Panel, consisting of K-12 practitioners and a small number of higher education faculty, and 
posting an online survey on the Department’s website. Phase 2, currently in progress, has involved 
Department staff refining revisions made in response to feedback gathered from stakeholders throughout 
Phase 1, creating first drafts of revisions, and consulting a group of content advisors in mathematics and 
ELA/literacy on those drafts. 


This report, along with summaries of proposed revisions prepared by ESE staff, represents the 
culminating activity of Phase 2. Taken together, this report and the summaries provide for the Board’s 
consideration a description of the process by which revisions were made, an overview of the proposed 
changes to standards and samples of the content of the revisions themselves. 


In Phase 3, to begin in November 2016, Department staff will present full drafts of the revised 
frameworks in ELA/literacy and mathematics and the Board will be asked to vote to release the revisions 
for a 60-day public comment period. At the end of the public comment period, the Department will 
incorporate public comments in its final revisions and present the final revised frameworks to the Board 
for a vote to adopt them. The graphic below summarizes the process and the following sections describe 
each component of these phases in more detail. 
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MA ELA/Literacy and Mathematics Standards Review Process 


Public Survey 


Open from February to mid- 
September 2016, the online 
survey captured public opinion 
Fl efolUie-laleMa-Tere)anlanl-lale(-10] 
revisions to the 2011 ELA and 
math standards. The responses 
helped to inform Standards 
Review Panel discussion. 
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Standards Review Panel 


20 ELA and 23 math panelists 
were selected to make 
recommendations for revisions 
to the 2011 standards. The 
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times from Februaryto July to 
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revisions to the standards. 
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Abt Associates 


20 PANELISTS 23 PANELISTS 


Content Advisory Group 


7 ELA and 6 Math content advisory 

groups each convened for one day 

to review and refine the proposed 
revisions to the 2011 standards 

recommended by the review 
panel. The content advisors also 
discussed the implications of the 
revisions for educators and 
students across the state. 


13 CONTENT AREA ADVISORS 


Professors from MA 
Colleges & Universities 


Expert Consultants 


District/School Leadership 
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standards to BESE for 
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comment (60 day) 
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and Math standards 
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2.1 Phase 1: Methods for Collecting Stakeholder Feedback 


Phase 1 began with the Department conducting outreach to establish both a statewide English Language 
Arts (ELA) and Literacy and Mathematics Standards Review Panel (Review Panel) and an online survey. 


2.1.1. English Language Arts (ELA)/Literacy and Mathematics Standards Review Panel 
Established 


In early January 2016, the Department solicited applications for participation in the English Language 
Arts/Literacy and Mathematics Standards Review Panel, whose members were charged with developing 
evidence-based recommendations to refine the 2011 ELA/Literacy and Mathematics standards based on 
lessons learned during Massachusetts’ implementation over the previous five years. To identify potential 
Review Panel members, the Department conducted outreach to various organizations and individuals 
across the Commonwealth, each of which is listed in Appendix A. In addition, the Department shared the 
invitation to apply to the Review Panel in the Commissioner’s “Weekly Update” in January and February, 
and other offices throughout the Department (e.g., STEM) sent notice of the opportunity via their 
listservs. The Commissioner solicited potential candidates through the Board, and received nominations 
from several organizations and individuals that had expressed interest in the standards review process. 


The Department reported that during the first three weeks of January 2016, it received applications and 
résumés from 126 individuals. The Department considered various criteria when selecting Review Panel 
members, including subject matter expertise and demonstrated experience in English language 
arts/literacy and math over the past five years. The Department also identified applications from educators 
with knowledge of working with special populations of students, such as English learners and students 
with disabilities. In addition, the Department took into consideration the grade levels with which Panel 
members had experience, to ensure a balance of expertise across grades: elementary (kindergarten 
through fifth grade), middle school (sixth through eighth grade), high school (ninth through twelfth 
grades), and higher education, and also considered some applicants with experience across several grade 
spans. In addition, the Department considered geographic representation from across the Commonwealth: 
Eastern, Central, and Western Massachusetts. The Department also sought members from a range of 
district types, including: urban, suburban/rural, charter and regional/vocational technical. 


ESE analyzed the pool of applicants to ensure members represented a balance of the categories described 
above and then reviewed individual résumés. The factors in the résumé review included: length of 
experience in schools, to ensure a mix of new and experienced educators; training and/or degrees earned 
in ELA/literacy or math fields; teaching and/or leadership experience in ELA/literacy and math; and prior 
engagement with Department initiatives such as MCAS development, standards development/revisions, 
and the Massachusetts PARCC Fellow Program’. Where possible, the Department sought out applicants 
who simultaneously represented several of the categories described above. 


* The PARCC Educator Leader Fellows Program was instituted in 2012 to use the expertise of approximately 40 
experienced P-16 educators in communications and professional development about the 2011 Curriculum 
Frameworks for Mathematics and English Language Arts and Literacy, the new question types and formats that 
would appear in the PARCC assessments between 2014 and 2016, and computer-based assessments. The 
program ceased in 2016 after the Board vote to develop the Next-Generation MCAS. In 2016, several former 
PARCC Fellows served on Department working groups, including the Standards Review Panel and the Panel on 
High School Testing. 
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The Department also applied exclusion criteria for individuals with any actual or perceived conflict of 
interest. In addition, ESE initially intended to select only Review Panel members who were not 
simultaneously participating in another work group related to the development of the Next-Generation 
MCAS, but that criterion was reconsidered and a few Panel members ultimately served on multiple work 


groups.’ 


In early February 2016, ESE selected 43 Panelists from across the state. Twenty Panelists brought 
expertise in ELA/Literacy and 23 in mathematics. A complete list of Review Panel members and their 
categories of experience can be found in Appendix B. Review Panel members consisted of 14 teachers; 
three principals/assistant principals; 18 department heads, coaches, directors or coordinators; one assistant 
superintendent and seven higher education faculty. Of these, 21 represented Eastern Massachusetts, 10 
represented Central Massachusetts and four represented Western Massachusetts. Ten Panelists were from 
urban districts, 21 were from rural or suburban districts, one was from a charter school, and one 
represented a regional vocational technical school. Nine Panel members had experience making revisions 
to previous versions of the Massachusetts Curriculum Frameworks or serving as PARCC Fellows for the 
Department. Review Panel members were informed of their charge to develop recommendations for 
refinements to the standards based on their own experiences, and based on responses to a statewide online 
survey (described below). 


2.1.2 Online Survey Created and Administered 


In addition to engaging stakeholder feedback through the Review Panel, in early February 2016, the 
Department created and posted an online survey to solicit public input on the ELA/Literacy and 
Mathematics Curriculum Frameworks. Aligned with the purpose of the Review Panel, the survey was 
designed to provide educators across the Commonwealth an opportunity to share lessons learned from 
implementing the 2011 ELA and mathematics standards over the previous five years. The survey asked 
respondents to respond with their opinions about the following design features of the 2011 standards: 


English Language Arts (ELA) and Literacy 


e Anchor standards 

e Coherent progressions across grades 

e The balance of reading, writing, speaking, and listening 
e Focus on informational text, research, and media 

e Focus on disciplinary literacy 

e College and career readiness for all students 


Mathematics 


e A balance of conceptual understanding, mathematical practices, and application 
e Coherent progressions across grades 

e Focus on select critical areas at each grade 

e College and career readiness for all students 


Other Next-Generation MCAS work groups included: Accessibility, Communications, High School Testing, 
and Test Administration. 
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The survey was designed to gather general information from educators familiar with the standards by 

inviting respondents’ input mainly via open responses. The survey asked respondents to provide basic 
demographic information and then invited them to answer five main questions about the standarts, as 

paraphrased below: 


1. Please identify which of the above design features you most value. Be as specific as possible in 
describing the implications of each design feature. 


2. Please identify the design features that have been most problematic in your school district's 
implementation of curriculum and instruction aligned to the standards. Be as specific as possible in 
describing the implications of each design feature. 


3. What content or skills in current standards, if any, do you consider inappropriate or unnecessary? 
Please include your reasoning. (If respondents answered this question, the next question asked 
respondents to list the specific standard(s). 


4. What content or skills, if any, do you consider to be missing in the 2011 standards? Please specify the 
grade level and provide your reasoning for why this content/skill is critical for all students to learn. 


5. Which standards, if any, could be modified to make them clearer or technically accurate? Please 
include your reasoning. If you choose to answer this question, the next question will ask you to 
provide the specific standard(s) to which you refer. 


A full version of the survey can be found in Appendix C. 


Beginning in mid-February, the Department sent announcements about the survey to the same groups that 
were contacted regarding the establishment of the Review Panel (listed in Appendix X). As part of the 
Next Generation MCAS initiative, the Department also established a Communications Workgroup that 
included representatives from several of these organizations, including the Massachusetts Teachers 
Association, the Massachusetts Charter Public Schools Association and the Massachusetts Association of 
School Committees. This workgroup met periodically with ESE staff to get updates on the standards 
revision process as well as the development of the statewide assessment. The Department invited 
participating organizations to forward information about and a link to the online survey and to conduct 
outreach encouraging their membership to participate. As with the solicitation for Review Panel 
applications, the Commissioner’s office included information about and a link to the survey in the 
Commissioner’s weekly update, as well as the “Teachers’ Top Three from ESE” email, through 
communications to the Board, through the Department of Higher Education channels, and through the 
listservs of offices with the Department (such as the STEM listserv and Urban Literacy Network). 


The survey was posted on the Department’s web site from February-early September 2016. In late June, 
near the end of Phase 1, Abt Associates conducted an analysis of the survey, coding responses and then 
using qualitative analysis software to analyze them. Once coded, Abt analyzed responses to each of the 

five questions and identified as themes those design features most frequently mentioned by respondents. 


Abt’s survey analysis found that by late June, 2324 respondents had viewed the online survey, while only 
364 respondents submitted complete or partially complete responses. Of these responses, 49% were from 
teachers, 33% from principals, coaches, superintendents and other administrators, 8% from parents and 


Abt Associates Final Report | pg. 13 


STANDARDS REVISION PROCESS 


4% from higher education professionals. Table 1 describes the types and response rates of respondents." 
Note that 186 (representing .002 percent of the state’s teachers) completed the survey; consequently the 
survey responses are not representative and should be interpreted in this context.” 


Table 1. Online Survey Respondents 


Partially 
Complete Complete 
Survey Respondents Surveys Surveys Total Surveys Percentage 
Teacher 176 10 186 48.9% 
Coach/Specialist 54 2 56 14.7% 
Administrator/Coordinator 40 1 41 10.8% 
Parent 29 2 31 8.2% 
Higher Ed Professional 16 0 16 4.2% 
Superintendent/Asst. 10 1 11 2.9% 
Superintendent 
Charter School Staff 11 0 11 2.9% 
Principal/Asst. Principal 8 1 9 2.4% 
Other 6 2 8 2.1% 
Business/Non-Profit Person u 0 7 1.8% 
School Committee Member 2 0 2 5% 
MA DESE Staff 2 0 2 5% 
TOTAL 361 19 380 100% 


Of the features that respondents described as most valuable, the most frequently mentioned were the 
ELA/literacy and mathematics standards’ coherent progression across grades (25% of responses), the 
balance of conceptual understanding (23% of responses), followed by the balance of reading, writing, 
speaking and listening in ELA/literacy (21% of responses). The most commonly referenced problematic 
features (12% of responses each) were the coherent progression across grades and college and career 


readiness. 


in late September, nine of whom submitted completed responses. 


An additional 61 respondents viewed the survey between Abt’s analysis in late June and the close of the survey 


In February and March 2016, ESE also administered the Views of Instruction, State Standards, Teaching, and 


Assessment (VISTA) Survey and invited responses from all superintendents and principals in the state as well 
as all teachers whose districts chose to participate. Of the five topics on which educators were invited to 
comment, the Massachusetts curriculum frameworks was one. Statewide, 21% of superintendents and 37% of 
principals responded to the VISTA survey. Responses were representative of the state, and reported results were 
statistically weighted to further ensure representativeness. Responses from superintendents and principals 
indicated that the vast majority of respondents agree somewhat or strongly that the curriculum frameworks 
prepare students for college and career, set appropriate expectations for learning, and stimulate instructional 
shifts so students meet expectations. Excerpts of relevant VISTA questions and responses can be found in 


Appendix D. 
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2.1.3. Review Panel Meetings 


Over the course of four different day-long meetings between February and July, members of the Review 
Panel met to discuss potential revisions to the standards. At these meetings, Panel members were asked 
by ESE staff to review and respond to suggested areas of revision from the survey responses received to 
date, correspondence from the public, and issues that the staff brought to the table based on their own 
expertise with the development, adoption, and implementation of the 2011 standards. The Department 
also invited Panel members to consider and develop proposed refinements to the ELA/Literacy and 
Mathematics Frameworks, drawing on their own expertise as educators. Before and between each 
meeting, ESE staff reviewed notes from the prior meeting and developed the agenda of topics that the 
Review Panel would consider. 


Meeting 1: February 24, 2016. The first Review Panel meeting was designed to provide an overview of 
the standards review process, define the charge of the Panel and develop a common understanding of the 
role of curriculum frameworks within a standards-based educational system. According to ESE 
documentation, staff from the Department provided a definition of a standard, described how standards 
are developed, summarized the organization of Curriculum Frameworks and disseminated copies of the 
current frameworks and cross-walks of the 2011 standards and previous standards (mathematics from 
2000 and ELA/Literacy from 2001), as well as other reference materials. ESE staff then led Review Panel 
members in several activities to help Panel members’ deepen their understanding of how the current 
ELA/literacy and math standards are structured and designed to address critical areas (math) or areas of 
emphasis (ELA/literacy) and how they progress through the grades. Staff from the Department also 
described the online survey and explained that at future meetings the Review Panel would be asked to 
review and react to feedback submitted via the survey. 


ESE staff also led a discussion with Review Panel members about the potential implications of making an 
addition or removing a standard on the vertical alignment of the standards and encouraged Panelists to be 
thoughtful about these implications in their review of proposed revisions and in their proposals for new 
revisions. Department staff then provided examples of revisions made to Common Core State Standards 
in another state as an illustration of the types of revisions Review Panel members might consider. Finally, 
staff from the Department explained to Review Panel members that they were charged with three main 
tasks in addition to recommending revisions: 1) to provide evidence to support any proposed changes, 2) 
to consider the impact of a change for the grade(s) below and grade(s) above, and 3) to consider the 
impact of any proposed change on district work on curriculum development and instructional change. 


Meeting 2: April 13, 2016. According to ESE documentation, at the second meeting, staff from the 
Department invited Review Panel members to review key themes from responses to the online survey 
(those received by April 6th) as well as to present their own topics for consideration. The Department 
categorized survey responses into three types: 1) suggestions for technical changes that the Department 
could propose to the Board, 2) substantive questions and feedback in which the Review Panel would be 
invited to engage, and 3) feedback outside the scope of the revision process or related to the standards 
implementation rather than standards content. The key themes that the Department presented from the 
survey were: there was overall support for the structure and design features of the standards, and there 
were some problems identified for consideration across grade levels in ELA/Literacy and mathematics. 


Department staff then invited Review Panel members to engage in a whole Panel discussion about 
disciplinary literacy, standards that appear in the 2011 ELA/Literacy Framework. The Department 
described disciplinary literacy as the ability to use “information in specific ways and purposes that are 
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relevant to and enhance understanding of a discipline, being able to read, write, and speak using 
disciplinary conventions and language, accessing disciplinary text and knowledge, and communicating 
using conventions of a discipline.” The K-5 standards include expectations for reading, writing, speaking, 
listening, and language applicable to a range of subjects, not limited to ELA. For grades 6-12, the 
ELA/literacy standards are divided into two sections, one specific to ELA and the other designed for 
history/social studies, science, and technical subjects. ESE asked Review Panel members to consider 
whether disciplinary literacy® should be incorporated into the math standards and then documented Panel 
members’ discussion and ideas. 


Informed by some of the survey responses, as well as Department priorities, ESE staff also developed 
specific questions or revisions for Panel members’ consideration by grade level in ELA/literacy and math. 
Department staff asked Panelists to divide into ELA and math groups and to further divide into grade 
level groupings of grades PK-5, 6-8 and 9-12 in order to respond to and discuss the following questions in 
English Language Arts/Literacy and mathematics. 


English Language Arts/Literacy Breakout Group Discussion 


Department staff developed the questions listed below (in italics) and facilitated discussions among 
Review Panel members divided into ELA subgroups and documented their deliberations (also included 
below) as well as their suggestions for revisions to the ELA/Literacy Framework documents. 


Table 2. ELA/Literacy Review Panel Recommendations 


Question Response 


PreK-5th Grades 


Are the ELA/literacy standards too much for Review Panel members discussed and generally agreed that no changes 
students in PK-5 in terms of their quantity and | were needed to PK-5 standards, explaining that many survey responses 
rigor? suggesting changes were not substantiated by Panel members’ review of 
the standards. Panel members did recommended moving the Grade 1 
language standard to produce and expand complete simple and compound 
declarative, interrogative, imperative, and exclamatory sentences in 
response to prompts (L.1.1.j) to 274 grade (2 L2.1f), to add complex 
sentences to grade 3, to move the 3" grade standard to “produce simple, 
compound, and complex sentences (L3.1i) to grade 1 and to add examples, 
especially in kindergarten and 1st grade. 


Are the language standards clear and The Review Panel reported that these standards were clear and appropriate 
appropriate, especially in the early and middle | but did recommend deleting the standard for kindergarteners to use the 
years? most frequently occurring inflections and as a clue to the meaning of an 


unknown word (L.K.4b), based on Panel members’ reasoning that this was 
not a pre-reading skill and was not needed for kindergarteners. 


Disciplinary literacy was a topic of discussion for the panels through July, as Department staff worked to 
incorporate panel feedback into the Frameworks revised draft. The revisions proposed for disciplinary literacy 
are reflected in the proposed recommendations in the draft Frameworks document being presented at the 
October BESE meeting. 
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Question 


Response 


Should the standards address the motor 
processes of writing more clearly or 
extensively? 


Does the progression of the Massachusetts- 
specific standard focused on identifying and 
analyzing literary works (R.MA.8A) require 
modification to make the genres less limited 
and/or the grade level expectations less 
specific? 


| Panel members cited research demonstrating the connection between 


handwriting and deeper comprehension and recommended keeping the 
standards related to writing by hand and extending those standards through 
grade 5. 


6th.8th Grades 


Panel members identified the following options: 1. omit this standard across 
all grade levels, 2. include it at certain grade levels only (for example, at 
grade 6), or 3. omit this standard but include elements of it in other 
standards. They came to general consensus around option 3. Panel 
members reported that this standard was not integrated into the other 
standards and suggested removing some of the specific requirements that 
they found redundant, unnecessary, or restricted to certain grades. 


Does the progression of the Massachusetts- 
specific standard focused on writing in 
different literary genres (W.MA.3A) require 
modifications to make the genres less limited 
and/or the grade level expectations less 
specific? 


Panel members agreed that this standard should be revised in some way 
because the genres listed in this standard were too specific, without strong 
rationale for why particular genres are placed at one grade level or another. 
They identified the following options: 

1. eliminate the standard completely as it is currently written; 

2. add its concepts to writing standards 10 or 4; 

3. consider adding a separate standard for writing poetry. 


Do the middle school reading standards 
require clarification in terms of providing a 
clear and coherent progression across grades, 
and providing clarity about the integration of 
various texts/media? 


Does the “Text Types and Purposes” cluster of 
writing standards require modification to 
prevent the standards from being too rigid in 
defining modes of writing? Do the standards 
relating to narrative writing require 
modification? 


Panel members recommended adding “cause and effect’ to the language in 
standard 3 across grades 6-8. They also recommended integrating standard 
7 across the grades, into information text and literature standards. Panel 
members recommended adding some examples (sample text sets) to 
illustrate standard 9 across grades. Finally, Panel members recommended 
that standard 2 in the Reading: Literature and Reading: Informational Texts 
strands include paraphrasing across grades 6-8. 

gth.42! Grades 


Review Panel members recommended combining standards 1-3 in a way 
that takes into account author's purpose and audience, that clarifies that all 
modes of writing (including narrative)are valued and expected to be taught. 


Do the standards for literacy in the content 
areas require modification to make clear the 
role of ELA teachers working with 
informational text related to other disciplines? 
Is additional guidance needed to support 
integration of literacy instruction across 
disciplines? 


Review Panel members noted that, in terms of their content, the standards 
make clear the role of ELA teachers working with the structure of 
informational text related to other disciplines, but recommended that 
standards for literacy in other subject areas should be taken out of the 
ELA/literacy framework and, if possible, inserted into the other subject 
areas’ frameworks. Panel members also recommended that 
speaking/listening should be included in disciplinary literacy standards for 
other subjects and perhaps language as well. They said that additional 
guidance was needed to help support the integration of literacy instruction 
across disciplines, noting that the development of this guidance was outside 
of the Panel’s charge. 
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Panel members were split on this topic. Some members recommended 
broadening the references so that historical texts in general were 
referenced but US history in particular was not. Others recommended that 
the original references to US foundational documents should be retained. 


Mathematics Breakout Group Discussion 


The Department developed the following questions and asked Review Panel members to respond to them, 
documenting Panel members’ responses and recommendations, as included below. 


Table 3: Math Review Panel Recommendations 


PreK-5t" Grades - 


Are there aspects of the PK-5 standards that 
can be reduced? For example, is the 
distributive property too challenging for grade 
3? Do line plots need to be included in each of 
grades 2-5? If not, which grades can they be 
removed from? Should the standard for whole 
number quotients (4.NBT.6) be removed from 
grade 4? Does the group recommend any 
additional reductions to the PK-5 standards? 


While consensus was not reached, some Review Panel members indicated 
that the distributive property is appropriate in 3rd grade but recommended 
Clarifying the expectations for the use of notation so that students not be 
required to produce an equation using parentheses. Some Panel members 
recommended eliminating the standards for line plots in grades 2-5. In 
addition, some Panel members recommended leaving standard 4.NBT.6 as 
is, and adding examples and strategies to help support teachers to 
understand the expectation and language of the standard. 


Are the K-3 standards missing some critical 
components? Should time and money be 
added to Kindergarten standards? Should 
patterns be added at the K-2 level? Should 
basic fraction terminology and unit fractions be 
added to grade 2? Should fractions of a set be 
explicitly added to grade 3? 


Are there aspects of the 6-8 standards that 
can be reduced? Should mean absolute 
deviation be eliminated from grade 6? Should 
systems of equations be moved to high 
school? 


Members of the Review Panel recommended that the concepts of time and 
money not be added to kindergarten standards. Some Panel members also 
recommended that patterning be considered as an addition in grades 1-2, 
(cautioning that specificity is extremely important) and language could be 
added to existing standards, perhaps adding patterning in standards for 
finding 10 more or less of a number without counting (1.NBT.5), determining 
odd and even numbers (2.0A.3) and skip counting (2.NBT.2). The Panel 
recommended adding text to the grade 2 standard for partitioning shapes 
(2.G.3) to partition a line into halves, thirds, etc. and labeling them with 
correct fraction notation such as 1/3 or 2/2. 

6-8!" Grades 
Review Panel members recommended that mean absolute deviation be 


eliminated from grade 6 and that standards related to systems of equations 
stay in grade 8. 
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Question 


Response 


Should adjustments be made to critical 
content in the grades 6-8 standards? Should 
students be expected to explain a proof for the 
Pythagorean Theorem in grade 8 or is 
clarification sufficient? Should conversions 
between fractions, decimals, and percents be 
added to the grade 6 standards? Should the 
current focus on ratios and unit rate at grades 
6-7 shift to a focus on rate? 


How can the scope of high school standards 
be reduced? Should some content be moved 
to a different course? Should the scope of 
Algebra | be reduced? If so, what changes do 
you recommend? Should concepts related to 
complex numbers and matrices be reduced or 
moved? Should the scope of statistics be 
reduced and/or moved to different courses? If 
so, what changes do you recommend? Should 
trigonometry be moved to Pre-Calc? Should 
geometric constructions be removed or moved 
to another course? If so, what changes do you 
recommend? Does the group recommend any 
other response to the comments on the 9-12 
standards? 


gth.42th Grades 


Some members of the Review Panel recommended that students should 
not be expected to explain a proof of theorem but rather show that they 
‘understand’ what the theorem is. Panel members also recommended not 
adding conversions between fractions, decimals, and percents to grade 6, 
but recommended a “bridge” between 4.NF.5, 4.NF.6 and 5.NF.3 that 
introduces decimals. Members of the Panel recommended that the 
Department consider including via a footnote or an example, additional 
denominators (2, 4, 5, 20, 25, 50) for equivalent fractions in the lower 
grades. Most Panel members recommended no change to the current 
standards focused on ratios and rate. 


Review Panel members did not recommend changes to the scope of 
Algebra | beyond moving find inverse functions (F-BF.4) to Algebra Il. Some 
members of the Panel also recommended that (+) Number and Quantity 
standards in Algebra II be removed. Panel members recommended that the 
statistics courses remain where they are, and some Panel members 
suggested removing all (+) Standards from Model Courses Alg. |, Geometry, 
Alg. Il, Integrated Math I, Il, and Ill. 


At the end of the April 13" meeting, the Department staff informed Panel members that the ESE team 
would examine the Panel’s recommendations and continue to make additional revisions aligned to that 
feedback. The Department then intended to share new, revised drafts of the Curriculum Frameworks 
documents with the Review Panel before the June 3™ meeting. 


Meeting 3: June 3, 2016. Just prior to the June an meeting, ESE staff sent Review Panel members drafts 
of the ELA/Literacy and Mathematics Curriculum Frameworks that incorporated revisions made during 
the April 13" meeting or revisions informed by discussions at the meeting. The goals of the June meeting 
were for Review Panel members to review and comment on the revisions proposed to date, consider the 
implications of these revisions, and propose new revisions if necessary. 


The Department began the meeting with an overview of the progress on revisions to date, then reviewed 
revisions common across ELA/literacy and mathematics. Department staff highlighted the importance of 
considering the impact of proposed revisions on the scope of the standards. Staff from the Department 
also reviewed the changes made to incorporate mathematics into the disciplinary literacy sections of the 
ELA/literacy standards and add a section on disciplinary literacy (though not adding standards) to the 


Mathematics Framework document. 
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Staff from the Department also called Panel members’ attention to standards that used the term 
“understand” and invited them to consider whether or not this term was clear, or should be changed to an 
“action verb that articulates the way we expect students to demonstrate ‘understanding.’” Department 
staff encouraged Panel members in each content area to examine the standards that used this term to 
better define how students would show that they understand a given concept or skill. 


As they did at the April meeting, the Department then asked Panel members to break into discipline- 
specific ELA/Literacy and Math groups to discuss their reactions to the most recent edits. 


ELA/Literacy Breakout Group Discussion 


The ELA/Literacy group began with a discussion of two proposals for revisions that were sent to the 
Department independent of the survey and were relevant to all grades. The Department staff summarized 
the proposals, invited discussion and documented Review Panel members’ corresponding 
recommendations. The first proposal, from the faculty of the English/Communications Department at a 
Massachusetts college, suggested the addition of four new anchor standards for reading, each focusing on 
aspects of rhetorical context. Panel members recommended integrating rhetoric into grade level/span 
standards because they described it as being underemphasized in the current standards (but emphasized in 
Advanced Placement Language courses) and being an important component of college readiness. 


The second proposal, from a member of the 2009-2010 Common Core State Standards writing team, had 
several recommendations: a) revise the PK-8 reading standards into grade bands of PK-K, 1-3, 4-5, 6-8, in 
order to avoid unintended specificity; b) revise Writing Standard 9 at grades 4-12 by removing examples, 
which refer to reading standards that call for comparing texts, as opposed to writing about a single text; c) 
change the sequence of Language standards on conventions of Standard English in middle school grades 
to make a more meaningful progression in teaching sentence structure, clauses, pronoun agreement and 
move some of the more advanced skills to high school; and d) to add the terms “subject” and “predicate” 
to an existing grade 4 language standard. Panel members were concerned that revising the PK-8 Reading 
standards would be difficult for districts to implement, and deliberated the recommendations to revise 
Writing Standard 9 and to change the sequence of the Language standards, as well as adding subject and 
predicate terms. 


Mathematics Breakout Group Discussion 


The Department asked Panel members to break into elementary, middle and high school groups and each 
group reviewed and discussed the revisions made to date by going through the line edits in the 
Frameworks document. The Department reviewed five of the main themes discussed at the April meeting 
and reported on which of these revisions they had chosen to incorporate in the proposed revisions 
included in the Frameworks document. The math group also engaged in a discussion of the terms 
“memorize” vs. “demonstrate fluency”, some Panel members advocated for more focus on problem- 
solving and critical thinking than memorizing, while others expressed concern that without having key 
math facts memorized, students would struggle as they moved into higher levels of math. Several 
members suggested defining fluency as a way to provide guidance for educators. 


At the end of the meeting, ESE staff gathered the full Review Panel together for a whole group discussion 
of potential implications of the proposed revisions on curriculum, instruction and assessment. 


Meeting 4: July 7, 2016. At this meeting, the Department provided an overview of the process to date, 
beginning with presentations from Deputy Commissioner Jeffrey Wulfson and Senior Associate 


Abt Associates Final Report | pg. 20 


STANDARDS REVISION PROCESS 


Commissioner Heather Peske, who described the next steps in presenting revised standards to the Board 
and the goal to have revised standards adopted by spring 2017. Abt Associates then provided a general 
analysis of survey data to identify the key themes. Abt reported that “clarifying language” was the most 
frequently requested change to the standards for both ELA/Literacy and math. Using polling technology, 
the Department then polled the 27 Review Panel members in attendance on the common topics that had 
been considered by the full Panel for revisions to the ELA/Literacy and math Frameworks and found that 
there was general, but not complete, consensus on the topics. The most frequently selected answers to the 
poll questions (representing a majority of responses from Panel members present) are described below: 


1. Are you ok with the scope of the standards as is, given edits made? The majority of panelists in 
attendance responded that they could support the edits: Eight Panel members responded that “the 
overall scope is what it needs to be,” and 14 responded they “would have liked to reduce, but can 
support the scope the way it is.” 


2. Do the strategies for enhancing disciplinary literacy work? The majority of Panel members reported 
that they supported the changes related to disciplinary literacy: 10 responded that the strategies were 
“a good mix and approach which will be helpful for the field” and eight responded that “the approach 
is better than we had but not ideal; let’s put it out for comment.” 


3. Does College and Career Readiness (CCR) need more attention? | Panel members’ responses 
indicated support for several options: 11 members responded that ““No, enough has been done with 
CCR and the standards are strong as is.” Seven Panel members supported the suggestion to “Just add 
visuals to illustrate how earlier grades support CCR” and six Panel members supported the suggestion 
that “it would also help to convene a group to discuss career perspectives and adjust the standards.” 


4. Should we provide guidance regarding social and emotional learning? Most Panel members in 
attendance supported the provision of guidance for social and emotional learning: 15 Panel members 
responded that “Yes, and the addition of a guiding principle is an appropriate way to do it, and six 
responded, “Yes, but guidance should go beyond a guiding principle.” 


5. Should we provide guidance regarding students significantly behind grade level? Panel members 
were split on the question of guidance for students significantly behind grade level: 11 Panel 
members responded “Yes, guidance in the Frameworks would be useful (strategy TBD).” And 16 
responded “No, Frameworks are not an appropriate avenue to address this challenge.” 


6. Should we provide guidance regarding technology access and use? Panel members were also divided 
on the question of guidance for technology: Nine Panelists responded “Yes, guidance in the 
Frameworks would be useful (strategy TBD)” and 14 responded “No, Frameworks are not an 
appropriate avenue to address this challenge.” 


The Department then invited Panel members to break into content area groups, but to stay in whole group 
discussions (rather than grade level groups) to review the proposed edits and to consider the implications 
of the proposed revisions as well as to think about any changes that would significantly disrupt school and 
district work. The discussions from the ELA/Literacy and math groups are summarized below. 
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English Language Arts/Literacy Breakout Group 


ELA/Literacy Panel members broke into their own group and ESE staff summarized the following 6 key 
considerations and possible edits that they had documented at previous Panel meetings, invited Review 
Panel members to consider and comment on these edits, and documented Panel members’ responses. 


1. Balance of Speaking, Listening, Reading, and Writing — ESE reported having added a Speaking 
and Listening Strand to Standards for Literacy in History/Social Studies, Science, Mathematics, and 
Technical Subjects 6-12. 


2. Coherent progressions across Massachusetts-specific standards in Reading and Writing — To 
improve coherence to reading, writing, and language standards, ESE merged R.MA.8 into RL.4.5 and 
6 in grades P-5; and into RL.7 at grades 11-12 — to allow for analysis of literary non-fiction. The 
Department subsumed W.MA.3 into anchor standard 4 and matching grade-level standards in grades 
P-12. The Department also adjusted Language Strands: L.1 in grades 1-3 for coherent progression and 
revised the chart for Language Progressive Skills, by Grade to reflect standards that need attention 
through the grades. 


3. Representation of Media — The Department changed S/L.2 anchor standard to read, “Integrate and 
evaluate information presented in diverse media and formats.” The Department also adjusted grade- 
level standards to match this anchor standard. W.6 at grades 9-12 replaces “message boards” with 
“discussion boards.” 


4. Inclusion of Literature — The Department changed “meaning or tone” to “meaning, tone, and mood” 
in grades 6-12 and included elements from R.MA.8 and W.MA 3 puts poetry into other standards. 


5. Sustained Reading and Writing — Reading Standard 10 refers to the range of reading and text 
complexity. In the 2011 ELA Framework, measures of text complexity were presented in 2- to 3-year 
grades spans (1.e., 2-3, 4-5, 6-8, 9-10, and 11-12). The Department revised text at PK-8 to indicate 
that students should be able to read texts at their individual grade level, rather than a grade band. It 
added a footnote with additional resources on measuring text complexity. 


6. Representing modes of writing — The Department added a footnote for Writing that explains the 
expectation that effective writing may blend elements of any of the three primary modes of argument, 
explanation, and narrative and added suggested resources for examples of effective writing. 


Mathematics Breakout Group 


In the math breakout group, ESE staff summarized the following seven key considerations and possible 
edits Department staff that had been proposed in prior Review Panel meetings, invited Review Panel 
members to review and discuss these edits, and then documented Panel members’ discussion and 
rationale. 


1. Balance of conceptual understanding, skills, and application of mathematical practices. Most 
Panel members described that there was enough balance in the standards of students building their 
conceptual understanding and skills and applying their knowledge. 


2. Focus on unit rate at grades 6-7. Panel members discussed a proposal from one member to increase 
the emphasis on rate in grades 6-7 as a means to help students better understand and solve proportion 
problems and prepare them for more complex concepts such as linear functions and slope. Some 
Panel members supported minor edits to make rate more explicit without impacting current standards. 
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3. Integration of statistics. Panel members discussed the scope of statistics at high school and 
considered reducing it or moving it to different courses. Most Panel members recommended leaving 
all the non-plus (+) statistics standards in Geometry and Algebra II and removing plus standards for 
statistics from typical 9" and 10" grade courses. 


4. Algebra Lin grade 8 for all students. Review Panel members considered a proposal from one Panel 
member to develop a pathway to allow middle school students to take Algebra I and progress through 
the highest levels of math (calculus) in high school. ESE proposed adding a guidance section to the 
Framework that describes different ways to compress the middle school standards to achieve Algebra 
I in grade 8 and/or provide different high school pathways to get students to calculus by grade 12. 
This guidance was originally produced in 2011 as a separate document to support schools and 
districts that wanted to address these issues. That document has been integrated into the revised 
Framework document and included in the revisions proposed for 2016. 


5. Reduction of high school scope. The Panel recommended that the scope of Algebra I should not be 
reduced but that all matrices standards be removed from Algebra II. The Panel also recommended 
that the information that appeared in footnotes in the 2011 standards be removed and integrated into 
the standards instead. 


6. Standards for high school calculus course. Review Panel members did not recommend adding 
standards for calculus. Some Panel members suggested that it might be redundant to develop different 
standards from the current AP Calculus standards. 


7. Memory or fluency for math facts and algorithms. Review Panel members engaged in a lengthy 
discussion about the difference between memory and fluency, without reaching consensus about 
revisions to the standards. Some Panel members called for more clarification of, and potentially the 
provision of definitions for, both memory and fluency. 


Final Scheduled Meeting 5: October 20, 2016. At this meeting, the Department will provide the Review 
Panel with a summary of the revisions being proposed to the Board. The Department planned this meeting 
as an opportunity to put the Phase 1 and Phase 2 activities in a broader context for the Panelists and 
identify how each of those activities contributed to the recommendations being brought forward to the 
BESE at their October meeting. It also provides an opportunity for the Department to report to the 
Panelists on Phase 2 activities and discussions and describe the likely steps remaining in the standards 
review process. 


2.2 Phase 2: Refine Revisions and Engage Content Advisors 


In Phase 2 of the standards review process the Department documented and summarized stakeholder 
feedback from Phase 1 and developed a systematic approach to vetting and refining the proposed 
revisions in preparation for Board consideration. To lead this phase of the process, in July 2016, the 
Department engaged two experienced and widely-respected former staff members, Barbara Libby and 
Susan Wheltle to lead the revision process in mathematics and ELA/Literacy, respectively. Barbara Libby 
is the retired Director of the ESE Office of Science, Technology/Engineering and Mathematics and Susan 
Wheltle is the retired Director of the Office of Literacy and Humanities. Libby and Wheltle led ESE’s 
previous frameworks writing and review panels in 1995-1997, 2000-2004, and 2007-2010. 


Libby and Wheltle, together with current members of the ELA/Literacy and Math staff from the 
Department, first reviewed the Phase | revisions proposed for mathematics and ELA/Literacy, 
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respectively. These ESE ELA/Literacy and mathematics teams then identified both major issues and 
detailed changes being considered for recommendation in each content area. 


In ELA/literacy, the Department team identified the following major categories in the revisions: 


e Integrated the content from two PK-12 Reading and Writing standards added by Massachusetts in 
2010 into other standards, thereby reducing the overall number of standards; 


e Edited standards to create greater clarity and smoother progressions in complexity, PK-12; and 


e Revised and expanded the glossary to provide a stronger resource for definitions of terms related to 
early literacy, writing, reading, language, and Standard English conventions. 


In math, Department staff identified the following major categories for possible revisions emerging from 
Phase 1 activities: 


e Improvements in clarity and inclusion of definitions for key mathematical terms to ensure consistency 
in the language of the standards; 


e The development of customized descriptions of the eight PK-12 Standards for Mathematical Practice 
by grade-spans, PK-5; 6-8; and high school level; 


e Additional guidance that presents middle and high school options for course sequencing, including 
pathways to Algebra I in grade 8 and to calculus by grade 12. 


2.2.1. Content Advisors 


As staff at the Department began to refine the revisions to standards that had been proposed by Panelists 
and survey respondents in Phase 1, they recognized the potential value of having this work reviewed by 
an additional set of advisors who possessed deep content area expertise as well as prior experience in 
reviewing state standards. Leading up to and after the June 3™ Review Panel meeting, some Panel 
members raised concerns that the cumulative effect of the revisions recommended across strands, clusters 
and grade levels, might have the unintended consequence of weakening the standards and/or disrupting 
their alignment across grades. The Department recognized that establishing a group of well-respected and 
experienced content advisors could help think through these concerns and provide new perspectives. 


In August 2016, the Department established two groups of content advisors, one for ELA/literacy and one 
for mathematics, whose members consisted of educators from Massachusetts public and private 
institutions of higher education, K-12 educators and individuals with experience supporting elementary 
and secondary educators’ instruction. Content advisors were identified by the Department with 
suggestions from the Executive Office of Education and other key stakeholders from the field. Advisors 
were selected based on the following criteria: ELA/Literacy and mathematics faculty from public and 
private colleges and universities in Massachusetts, experience working in Massachusetts public schools 
and/or with Massachusetts public educators in ELA/Literacy and mathematics, and knowledge of state 
standards and/or experience in prior standards revision processes. A list of content advisors is provided in 
Appendix E. 


The content advisors met for two separate day-long meetings (one for ELA/literacy and one for math) on 
September 15" and 16", 2016. The content advisors were asked to review draft revisions and draw on 
their own content expertise and professional experience to provide guidance and suggestions. Advisors 
were also asked to consider the following guiding questions about the proposed changes emerging from 
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the review panel: Is the cumulative effect of the recommended edits an improvement in the clarity, 
coherence, and rigor of the Framework? If not, what specifically do you recommend we do differently? 


The meetings were facilitated by Susan Wheltle (ELA/Literacy) and Barbara Libby (mathematics). Both 
groups of content advisors were provided with documents several weeks in advance that summarized the 
proposed revisions to date. As described in more detail below, the facilitators structured the discussions 
around topics upon which there was not consensus among survey responses and Review Panel feedback. 
Convening the content advisors then provided an opportunity for the Department to gather new, relevant 
perspectives on these topics to further inform their upcoming revisions. 


The Department documented the content advisors discussions and recommendations, and identified 
common themes among the recommendations from both ELA/Literacy and math content advisors. One 
common theme was that both groups were generally positive about the standards and agreed with 
Panelists and survey respondents that what was most needed was clarifying language, as well as 
additional examples and resources to help educators with implementation of the standards. Both groups of 
content advisors cautioned about incorporating too many examples or explanatory materials within the 
standards themselves and suggested the possibility of using technology in new ways to make the 
standards more interactive and accessible, to make stronger connections among the Frameworks’ sections, 
and to include links to existing and new resources useful for implementation. Both ELA/Literacy and 
math content advisors also suggested developing guidance and resources to support districts in planning 
for implementation of the revised (2017) Frameworks. In addition, each group of advisors had content 
specific recommendations, a summary of which is provided below. 


September 15, 2016 — English Language Arts/Literacy Content Advisor Meeting. In mid-August, the 
Department sent content advisors three documents summarizing the proposed revisions: 1) an Overview 
of Major Issues and Recommendations for the Massachusetts ELA/Literacy Framework Standards 
Review 2016, 2) a Summary of Detailed Issues and Recommendations for the Massachusetts 
ELA/Literacy Framework Standards Review 2016, which included specific standards identified for 
proposed revisions and a rationale for the revisions, and 3) Draft Massachusetts Curriculum Framework 
for ELA/Literacy (as of August 18, 2016) with tracked changes and highlights. The Overview of Major 
Issues document included six main issues, listed below, which Susan Wheltle used to frame the 
discussion with advisors: 


Literature and sustained reading 

Writing 

Developmental appropriateness of PK-8 standards, PK-12 progressions, and terminology 
Literacy in subject areas other than ELA 

Alignment with other BESE and DESE initiatives 

The cumulative effect of the recommended edits 


OY Pe GN 


ELA/literacy advisors discussed each topic area, weighed the trade-offs of making or not making the 
contemplated revisions and made the following over-arching recommendations: 


e Maintain the strength of the standards. The advisors regarded the 2011 Frameworks as having 
been carefully designed and stated that it was critical to maintain the focus, coherence, rigor and 
structure of the 2011 framework. 
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Improve clarity of some standards. Advisors suggested making edits to improve the clarity of the 
standards’ language or to clarify expectations. They also recommended adding examples to the 
standards, and, in some instances, definitions to the glossary as ways to improve clarity. 


Support implementation. Advisors recommended using existing ESE resources published since 
2011 and linking those resources and guidance documents to digital versions of the new Frameworks. 


Improve information about text complexity. Advisors emphasized the importance of introducing 
students to complex texts across the grades and suggested including more detailed information on 
measuring text complexity in the explanatory materials accompanying the Reading Standards. 


Preserve strong progressions in Reading Literature Standards. Content advisors recommended 
maintaining a strong emphasis on knowledge of literary concepts, either through retaining the 
Massachusetts standard or incorporating its concepts in the Reading, Writing, and Language 
standards. 


Highlight integration in types of writing. Consistent with revisions the Department had made in 
response to feedback from the survey and Review Panel discussions, advisors recommended 
including more detailed information on the ways in which effective writing often blends argument, 
explanation, and narrative to the Introduction to the Writing Standards, as opposed to the way the 
current standards make the three main types of writing seem separate and distinct. 


Make stronger connections among the Language, Writing and Reading standards. Advisors 
recommended simplifying the grade level Language Standards and emphasizing their relationship to 
interpreting texts and writing. Advisors suggested making it clearer that knowledge of Standard 
English conventions (grammar) is assessed in the context of writing and also recommended 
consolidating glossary entries related to grammar and placing them in the Language section of the 
Frameworks document. 


Encourage teaching of literacy in subjects other than English. Advisors supported the 
recommendation to include literacy in other subject area Frameworks. 


September 16, 2016 —- Mathematics Standards Review Content Advisors Meeting. In late August, the 
Department sent math content advisors three documents summarizing proposed revisions: 1) an Overview 
of Major Issues and Recommendations for the Massachusetts Mathematics Framework Standards Review 
2016, 2) a Summary of Detailed Issues and Recommendations for the Massachusetts Mathematics 
Framework Standards Review 2016, which included specific standards identified for proposed revisions 
and a rationale for the revisions, and 3) Draft Massachusetts Curriculum Framework for Mathematics (as 
of August 27", 2016) with tracked changes and highlights. ESE staff identified six main discussion topics 
for content advisors: 


DN ON ea 


High School (+) Standards 

Options for Course-taking Pathways and High School Statistics and Calculus 
Mathematical Practice Standards 

Rates, Ratios, and Proportional Reasoning 

Consistent, clear language 

The Cumulative Effect of the Recommended Edits 
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Math content advisors discussed each topic area, weighed the trade-offs of making or not making 
revisions and made the following recommendations, the first three of which are aligned to the 
ELA/Literacy advisors’ recommendations: 


e Maintain the strength of the standards. As with the ELA/Literacy content advisors, the 2011 
Frameworks were regarded as having been carefully designed and the advisors stated that it was 
critical to maintain the focus, coherence, rigor and structure of the 2011 framework. 


e Improve the clarity of some standards. Again, similar to the feedback from the ELA/Literacy 
advisors, math advisors suggested making edits to improve the clarity of the standards’ language or to 
clarify expectations. They also recommended adding examples to the some standards and/or 
definitions to the glossary to improve clarity. 


e Support implementation. Advisors recommended using existing ESE resources published since 
2011 and linking those resources and guidance documents to digital versions of the new Frameworks. 


e Keep the Math Pathways guidance document in the revised Framework. Advisors recommended 
modifying this document to expand the options for course taking pathways, noting that the current 
pathways all lead to calculus. Advisors suggested adding pathways leading to Advanced Quantitative 
Reasoning, Pre-Calculus and Statistics. 


e Keep the (+) standards in the Model High School courses. Advisors reasoned that these standards 
help maintain the level of rigor of the 2011 Framework and suggested including a clear explanation of 
why the (+) standards are included in the Framework (i.e. they provide options for differentiation, 
ensure equity and prepare students who are planning to take advanced courses or pursue degrees or 
careers for which this content is a prerequisite. 


e No model statistics or calculus course needed now. Math content advisors suggested instead 
providing a link to the AP Calculus syllabi. 


e Add guidance and rationale for statistics. Advisors recommended including a narrative that 
explains the importance of statistics, particularly in the context of our modern world and noted that 
there is already a set of statistics standards in each of the model courses. 


e Keep current language for “fluently” and “know by memory.” Content advisors expressed a 
unanimous opinion that knowing from memory was not to be done in isolation, but along with the 
development of conceptual understanding of numbers, quantity and number sense as well. 


e Clarify Ratio, Rate and Proportional Reasoning Progression in grade 6-7. Content advisors 
recommended adding rate examples to clarify this progression. 


e Redesign the grade-span descriptions of the 8 Standards for Mathematical Practice. Advisors 
suggested developing a chart that lists each Practice and provides descriptions of how the content and 
practices can be connected to better see the progression of content and practices through the grades. 


2.2.2 Conclusion of Phase 2 


The Department’s ELA/Literacy and mathematics standards review teams have incorporated the content 
advisors’ feedback , as well as feedback gathered through Phases | and 2, into the summaries of revisions 
that they are presenting to the Board at the October meeting. These summaries will be presented in two 
documents: one that presents major issues and one that presents the details by grade level with the 


Abt Associates Final Report J pg. 27 


STANDARDS REVISION PROCESS 


original and revised wording of the standard followed by the rational for the change. These summary 
documents, , therefore, will reflect the work of ESE’s ELA/Literacy and mathematics standards review 
teams, led by Wheltle and Libby, to synthesize and incorporate the deliberations of the Review Panels and 
content advisors conducted during Phases 1 and 2 of the review process. 


As ESE incorporated the feedback received through Phases 1 and 2 and prepared summaries for the 
Board, Department staff focused on four key areas of improvement: focus, clarity, coherence and rigor. 
Past iterations of standards reviews have also emphasized these areas. In the 2016 revisions, the 
Department is aiming to improve the focus by ensuring attention to critical areas at each grade level and 
to improve the clarity of standards through making wording more precise to provide clear guidance to 
educators. ESE intends to improve coherence by making careful adjustments to strengthen the 
connections within and across grades. To improve rigor, the Department is focusing on improving the 
skills and knowledge required for Massachusetts students to succeed in college and careers. 


The recommendations selected for presentation to the Board have been consistent discussion topics raised 
through Phases 1 and 2. For both content areas, the proposed revisions respond to suggestions, concerns 
or questions raised by stakeholders. In English language arts/literacy, the Department has identified the 
issue of “reading closely and writing about complex texts” as the major recommendation that they will 
describe to the Board. In mathematics, the major revision that will be presented by the ESE team will be 
“options for course-taking sequences.” Both the ELA/literacy and mathematics teams will also present 
examples of proposed revisions intended to increase the coherence, focus, rigor and clarity of the 
standards. 


2.3. Phase 3: Soliciting Public Comment and Adopting Revisions 


The 2016 standards revision process, while not a wholesale revision of the standards, has provided a 
strategic opportunity to make improvements to the standards not possible, or not recognized in 2011. In 
Phase 3, The Department will incorporate Board feedback on proposed revisions from BESE’s October 
meeting and will present drafts reflecting that feedback for the Board’s November vote on whether to 
release the drafts of the Frameworks documents for public comment. From December through March or 
April, the Department will conduct a public comment period, synthesize public comment, and make final 
revisions, resulting in final ELA/literacy and Mathematics Frameworks documents submitted to the Board 
for adoption. 
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In supporting the work of ESE on behalf of the Board, Abt Associates conducted a cross-state analysis of 
revisions to the Common Core State Standards (CCSS) made in nine other states. This analysis was 
conducted to provide ESE staff with additional context for Massachusetts’ anticipated revisions. Abt 
initially conducted a broad scan of the standards review processes conducted across states that adopted 
CCSS, and ultimately settled on a more detailed review of the revised standards in nine states (AL, AR, 
CA, FL, GA, MS, NJ, OH, and UT) that, like Massachusetts, had revised the Common Core State 
Standards.’ To inform the revisions that ESE was considering to specific standards, Abt also provided 
ESE with targeted analyses of particular Massachusetts ELA/literacy and math standards identified for 
revision in Phases 1 and 2, and highlighted the revisions made (and not made) to these standards in the 


nine states listed above. 


For the general cross-state analysis, Abt reviewed the mathematics and the ELA/literacy revisions in these 
nine states and coded every revision made in every grade in ELA/literacy and mathematics according to 
the type of change made. The table below describes the codes used to categorize each revision. 


Table 4. Standards Revision Categories 


Category Definition 
new standard added a new standard 
deleted standard deleted a standard 
addition added a concept or skill to an existing standard 
deletion removed a concept or skill from an existing standard 


split standards 


split skills from one standard into multiple standards 


combined standards 


combined skills from multiple standards into one standard 


moved lower 


moved standard to a lower grade 


moved higher 


moved standard to a higher grade 


moved cluster 


moved to another cluster within the same grade level/subject area (9-12) 


changed example 


made changes to examples or "e.g." text 


clarification 


Clarified text; changed wording, formatting or notes 


We selected these states by eliminating the 4 states (AK, NE, TX, VA) that never adopted Common Core State 


Standards, the 3 states (IN, OK, SC) that reported having repealed or withdrawn from CCSS, the 6 states (LA, 
MI, MO, PA, TN, WY) in which public accounts indicated that states intended to or had begun a process to 
repeal or replace CCSS, the 8 states (AZ, ID, IA, KY, MT, NY, ND, SD) undergoing a standards revision 
process still underway by August 31, 2016, and the 20 states (CO, CT, DE, DC, HI, IL, KS, ME, MD, MN 
(ELA only), NV, NH, NM, NC, OR, RI, VT, WA, WI, WY) that have made no revisions to CCSS to date. 


Abt Associates 
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Mathematics 


Overall 26.5% of mathematics standards across the nine states were revised. No changes were made to 
73.5% (2,548) of the standards. The number of standards revised or added by these states ranged from a 
17 to 282. Additionally, across the nine states, eight states added 51 total new standards. Some states 
added standards for new courses. For example, three states added calculus standards and one state added 
courses for Algebraic Connections, Discrete Mathematics, Mathematical Investigations, and Analytical 
Mathematics standards. 


Among the revisions made in these nine states, we found that the majority (68%) of the math revisions 
were clarifying changes in which states revised standards’ wording, formatting or notes or made changes 
to the examples provided. The next most common type of revision (25%) was to add a concept or skill to 
a standard. Figure 1 shows the percentage of mathematics standards that were revised and not revised, 
and, among the revised standards, the percentages of each type of revision made across the nine states 
reviewed. 


Figure 1. Nine-State Analysis of Mathematics Standards Revisions 


Moved cluster, 
0.0% 
Moved higher, 
0.5% 
Combined 
standards, 0.0% 


Moved lower, 0.0% 


Deleted standard, 
1.3% 


Split standard, 0.2% 
Deletion, 5.6% 
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English Language Arts 


In ELA, across the nine states reviewed, the average number of changes made to standards was 102 and 
the number of standards revised by each state ranged from 12 to 330 of total standards. We found that 
2,851 (76.8%) ELA standards were not revised and 794 (23.2%) were revised. 


The majority (69.0%) of the ELA revisions were clarifying changes. Adding a concept or skill to a 
standard was the next most common revision (24.8%). In addition to revising standards, states also added 
a total of six new ELA standards, with states ranging from adding zero new standards to adding three new 
standards. Figure 2 below shows the breakdown of changes by type of revision to the ELA standards. 


Figure 2. Nine-State Analysis of ELA Standards Revisions 


Moved cluster, 
0.0% 
Moved higher, 
0.1% 
Moved lower, 0.2% 
Combined 
standards, 0.0% 


Split standard, 
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On balance, the large majority of changes made in these nine states focused on clarifying the standards 
already in place. The revisions suggest that states mainly retained the original standards as adopted. 
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Review Panel and Survey 


Individuals: 


e MA state senators, representatives, and their aides 

e BESE members 

e ESE staff members 

e Next-Generation Workgroup applicants (accepted and rejected) — Accessibility, Communications, 
ELA/math panel, High School Testing, Test Administration 

e PARCC Fellows (43 K-12 and higher ed faculty who supported PD around the PARCC assessment) 

e Science Ambassadors 


And the following organizations: 


e American Federation of Teachers Massachusetts 

e Arts|Learning 

e Association of Teachers of Mathematics in Massachusetts 

e Center for Applied Special Technology 

e Center for Education Policy Research 

e Citizens for Public Schools 

e Citywide Parent Council (Boston) 

e Citywide Parent Planning Advisory Council (Worcester) 

e Federation for Children with Special Needs 

e Massachusetts Administrators of Special Education 

e Massachusetts Advocates for Children 

e Massachusetts Afterschool Partnership 

e Massachusetts Association for Curriculum Development 

e Massachusetts Association of 766 Approved Private Schools 
e Massachusetts Association of School Committees 

e Massachusetts Association of School Superintendents 

e Massachusetts Association of Science Teachers 

e Massachusetts Association of Teachers of Speakers of Other Languages 
e Massachusetts Association of Vocational Administrators 

e Massachusetts Business Alliance for Education 

e Massachusetts Charter Public Schools Association 

e Massachusetts Computer Using Educators 

e Massachusetts Council for the Social Studies 

e Massachusetts Cultural Council 

e Massachusetts Department of Early Education and Care 

e Massachusetts Department of Higher Education 

e Massachusetts Education Technology Administrators Association 
e Massachusetts Elementary School Principals’ Association 


e Massachusetts Foreign Language Association 
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e Massachusetts Office for Refugees and Immigrants 

e Massachusetts Office of Information Technology 

e Massachusetts Organization of Educational Collaboratives 

e Massachusetts Parent Teacher Association 

e Massachusetts Reading Association 

e Massachusetts School Counselors Association 

e Massachusetts Science Education Leadership Association 

e Massachusetts Secondary School Administrators’ Association 
e Massachusetts Teachers Association 

e Massachusetts Technology Education/Engineering Collaborative 
e Massachusetts Writing Project 

e National Center on Time and Learning 


e New England Association of Teachers of English 
e Pioneer Institute 

e Rennie Center for Education Research and Policy 
e Stand for Children 

e Massachusetts Digital Learning Advisory Council 
e Massachusetts Student Advisory Council 

e Teach Plus 

e WGBH Educational Foundation 
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Appendix B: Massachusetts ELA/Literacy and Mathematics Standards Review Panel 


Other 
Expertise Area District/School Role Setting Qualifications 
z 
a = 3 
3 a i = || Wic alllres => & 
2 Bs 23 fs £5 2|¢ 
@ | s /BEiS#8| 22 | 3 Salaa| = ae 
Review Panelist Name si 8, 2 EE lacg2| ee & Location = a S 2 Level Se 
Rachel Barlage mG we East v High 
Eileen McQuaid v v East Middle v v 
Tricia Clifford v v ve ue East ce Elem. 
Linda Crockett v v West v Middle v 
Lisa Dion ve v East ve Elem./ 
Middle/ 
School 
Oneida Fox Roye v v East v Elem. v 
Andrea Gobbi a v East v High 
Donna Goldstein v v East v Middle v 
Andrea Griswold a ve West v Middle 
Susan Hehir v v Cent. v Elem. 
Anna Hill ve A “a Cent. we Middle 
John Kucich ve v East High. Ed 
David Langston v v West High. Ed 
Danika Ripley v v East v Elem. a v v 
Melissa Ryan v va a East a Middle 
Karyn Saxon a v Cent. a Elem. 
Kathleen Tobiasson Ma v Cent. A Middle/ 
High 
Nancy Verdolino ve v Cent. v Elem. 
Rob Whitman v ve East High. Ed 
Kerry Winer < v East v Middle 
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Jennifer Berg 0 v Cent. High. Ed 
Tara Brandt ve v West Elem. 
Linda Dart-Kathios e v East High. Ed 
Linda Davenport v v East Elem. a 
Beth Delaney v e East Elem. 
Tom Fortmann v N/A NIA v 
Sarah Hopson v v West v Elem. v 
Nancy Johnson v a v Cent. a High 
Patty Juranovits v v v East Elem. 
Elizabeth Kadra oA G Cent. a Middle 
Patrica Kavanaugh v v East a Middle 
Stephanie Lowe a East Middle 
Jim McCleary ~ v Cent. Middle/ 
High 
Linda McKenna v v Cent. a Elem. 
Raigen O'Donohue v A East v Elem. 
Eileen Perez v v Cent. High. Ed 
Laura Raposa v a Cent. ¥ Elem. ul 
Heather Ronan ve e East Elem v 
Fran Roy v v East Elem 
Jeffrey Strasnick v East v Elem 
Brian Travers v v East High. Ed 
Meghan Walsh v v Cent. v Elem v 
Joanne Zaharis v v East Elem 


Abt Associates 
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Appendix C: Massachusetts ELA and Math Standards Review Online 


Survey 


Massachusetts ELA and Math Standards Review Feedback 
Lessons learned from implementing the 2011 ELA and mathematics standards over the past 5 years 


The 2011 standards were designed to achieve the following features: 


English Language Arts (ELA) 


Anchor standards 

Coherent progressions across grades 

A balance of reading, writing, speaking, and listening 
Focus on informational text, research, and media 
Focus on disciplinary literacy 

College and career readiness for all students 


Mathematics 
e A balance of conceptual understanding, mathematical practices, and application 
e Coherent progressions across grades 
e Focus on select critical areas at each grade 
e College and career readiness for all students 


Survey Questions for K-12 Educator/Teacher/Administrator, Parent, Student, or Other 


1. Please identify which of the above design features of the 2011 ELA and mathematics standards 
you most value after your school district implemented curriculum and instruction aligned to the 
standards. Be as specific as possible in describing the implications of each design feature. 


2. Please identify the design features of the 2011 ELA and mathematics standards that have been 
most problematic in your school district's implementation of curriculum and instruction aligned to 
the standards. Be as specific as possible in describing the implications of each design feature. 


3. What content or skills currently included in the 2011 standards, if any, do you consider 
inappropriate or unnecessary (you recommend taking out that content or skill)? Please include 
your reasoning. If you choose to answer this question, the next question will ask you to provide 
the specific standard(s) to which you refer. 


3a. (If answered Question 3) Given your answer to the previous question, please list the specific 
grade-level standard(s) to which you refer (e.g. "ELA Reading Standards: Foundational Skills, 
Grade 2, #3b") 


4. What content or skills, if any, do you consider to be missing in the 2011 standards (you 
recommend adding that content or skill)? Please specify the grade level and provide your 
reasoning for why this content/skill is critical for all students to learn. 
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5. Which standards, if any, could be modified to make them clearer or technically accurate? Please 


include your reasoning. If you choose to answer this question, the next question will ask you to 
provide the specific standard(s) to which you refer. 


5a. (If answered Question 5) Given your answer to the previous question, please list the specific 
grade-level standard(s) to which you refer (e.g. "ELA Reading Standards for Literature, Grades 9- 
10, #7"). 


6. Please describe your experience with the standards and your ideas on how to improve them. 


Survey Questions for Higher education professor/administrator 


1. 


Please identify which of the above design features of the 2011 ELA and mathematics standards 
you most value in support of college and career readiness to the degree that they prepare students 
to be successful in a college entry-level credit course. Be as specific as possible in describing the 
implications of each design feature. 


Please identify the design features of the 2011 ELA and mathematics standards that have been 
most problematic in support of college and career readiness to the degree that they prepare 
students to be successful in a college entry-level credit course. Be as specific as possible in 
describing the implications of each design feature. 


What content or skills currently included in the 2011 standards, if any, do you consider 
inappropriate or unnecessary in support of college and career readiness to the degree that they 
prepare students to be successful in a college entry-level credit course? Please include your 
reasoning. If you choose to answer this question, the next question will ask you to provide the 
specific standard(s) to which you refer. 


3a. (If answered Question 3) Given your answer to the previous question, please list the specific 
grade-level standard(s) to which you refer (e.g. "ELA Reading Standards: Foundational Skills, 
Grade 2, #3b") 


What content or skills, if any, do you consider to be missing in the 2011 standards (you 
recommend adding that content or skill)? Please specify the grade level and provide your 
reasoning for why this content/skill is critical for all students to learn. 

Which standards, if any, could be modified to make them clearer or technically accurate? Please 
include your reasoning. If you choose to answer this question, the next question will ask you to 
provide the specific standard(s) to which you refer. 


5a. (If answered Question 5) Given your answer to the previous question, please list the specific 
grade-level standard(s) to which you refer (e.g. "ELA Reading Standards for Literature, Grades 9- 
10, #7"). 


Please describe your experience with the standards and your ideas on how to improve them. 
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Appendix D: VISTA Survey Questions and Responses Related to 


Curriculum Frameworks 


Superintendent Responses 


Please indicate the extent to which you agree or disagree with the following statements about the 
Massachusetts Curriculum Framework. 


The ELA Curriculum Framework clearly indicates the content teachers 
should teach. (N=68) 


The ELA Curriculum Framework sets appropriate expectations for student 
learning at each grade level. (N=68) 


The ELA Curriculum Framework positively affects the degree to which 
students are prepared for college and career. (N=65) 


The Mathematics Curriculum Framework clearly indicates the content 
teachers should teach. (N=67) 


The Mathematics Curriculum Framework sets appropriate expectations for 
student learning at each grade level. (N=68) 


The Mathematics Curriculum Framework positively affects the degree to 
which students are prepared for college and career. (N=67) 


0% 20% 40% 60% 80% 100% 


MDisagree Strongly mDisagreeSomewhat © Agree Somewhat Agree Strongly 
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Principal Responses 


Please indicate the extent to which you agree or disagree with the following statements about the 
Massachusetts Curriculum Framework. 


The ELA Curriculum Framework sets appropriate expectations for student 
learning at each grade level (N=250) 


The ELA Curriculum Framework positively affects the degree to which 
students are prepared for middle school/college and careers (N=246) 


The ELA Curriculum Framework positively affects how well students are 
prepared to compete in the workforce (N=250) 


Appropriate expectations for student learning at grade level - Math 
(N=208) 


The Math Curriculum Framework: Positively affects the degree to which 
students are prepared for middle school/college and careers (N=215) 


Math Curriculum Framework: Positively affects how well students are 
prepared to compete in the workforce (N=241) 
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APPENDICES 


ELA Content Advisors 
William Amorosi v a 
Mary Ann Cappiello « v Lesley University Private High. Ed ~ 
Erika Thulin Dawes % v Lesley University Private High. Ed 
Lorretta Holloway ve v Framingham State University Public High. Ed a 
Brad Morgan a Essex Technical High School Voc. Tech High School 
Deborah Reck v a 
Jane Rosenzweig ve ce Harvard University Private High Ed. v 


Math Content Advisors 


Richard Bisk v v Worcester State University Public High Ed. 

(emeritus) 
Andrew Chen v v | EduTron Corporation ¥ 
Al Cuoco v v | Education Development Center 
Sunny Kang v v Bunker Hill Community College Community High Ed. 
Maura Murray a v Salem State University Public High Ed. ve 
Kimberly Steadman ae Edward Brooke Charter School Charter School Elem./ Middle 
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